Skip to content

.NET: Give a hosted agent a single source of conversation history - #7525

Merged
Roger Barreto (rogerbarreto) merged 18 commits into
microsoft:mainfrom
rogerbarreto:features/hosted-chat-history-provider
Aug 7, 2026
Merged

.NET: Give a hosted agent a single source of conversation history#7525
Roger Barreto (rogerbarreto) merged 18 commits into
microsoft:mainfrom
rogerbarreto:features/hosted-chat-history-provider

Conversation

@rogerbarreto

@rogerbarreto Roger Barreto (rogerbarreto) commented Aug 5, 2026

Copy link
Copy Markdown
Member

Motivation & Context

A hosted agent quietly kept a second copy of its conversation.

Two different things record a hosted turn. The AgentServer SDK's storage provider records it
around the container's handler, and that record is the conversation the caller reads back.
Separately, the service behind the agent's own chat client records the turn again whenever the
agent runs with storing left on, onto a trail of its own. Nothing reads that second trail, and
nothing reconciles it with the first.

The same conversation could also reach the model twice, once as input from the platform's record
and once replayed by the agent itself, consuming extra tokens for no gain.

Description & Review Guide

  • What are the major changes?

    One source of history, and one recording of each turn.

    Change Why
    History always comes from the platform's own record (ResponseContext.GetHistoryAsync) and is passed as input, marked as chat history one source per turn, and the marking stops those messages being written back as if they were new
    For the length of one run, the agent's chat history provider is replaced by VolatileChatHistoryProvider, which holds messages in a field and is dropped when the run ends the agent cannot serve a second copy of the conversation from a longer lived store, and cannot write into one either
    The chat options built for the run always ask the agent's own service not to store the turn, on both OpenAI request shapes (CreateResponseOptions and ChatCompletionOptions) no second recording downstream. The agent's own factory is invoked first and its result is what gets the setting, so whatever the container configured survives
    A session carrying a conversation id is refused up front with HTTP 400 service_managed_chat_history_not_supported a conversation id means the agent's service is keeping a conversation of its own. Refusing plainly beats failing halfway through a turn
    AgentSessionStore.GetSessionAsync now returns null when nothing is stored, and GetOrCreateSessionAsync is added alongside it the handler can tell a resumed session from a fresh one without reading state that the handler itself writes
    History is withheld only when resuming a hosted workflow a workflow session already carries the earlier turns in its checkpoint, so replaying them would re-drive completed steps

    What a single turn looks like now:

graph LR
    C["Caller"] -->|"turn, store as the caller asked"| P["AgentServer storage provider<br/>records the turn"]
    P --> H["Container handler<br/>reads that record back as the input history"]
    H -->|"store off"| S["The agent's own service<br/>answers and records nothing"]
    S --> P
Loading
  • What is the impact of these changes?

    Container Before Now
    Ordinary Foundry ChatClientAgent, as in the first hosted agent sample every turn also stored downstream, and history could reach the model twice the turn is recorded once, by the platform
    Agent with a chat history provider of its own (Cosmos, Valkey, in memory) that store received the platform's turns as well and grew a parallel copy the provider is stood down for the run, so the store is left alone
    Agent whose chat client points at a service that keeps the conversation silently ran with two conversations refused with a 400 that says what to change
    Hosted workflow unchanged unchanged

    A new integration test covers this against a live Foundry project. The container agent is an
    ordinary ChatClientAgent, wrapped so that after each run it reports back the conversation its
    own run left behind, and the test goes looking for it on the service. Finding it means a second
    copy exists.

    AgentSessionStore here is the one in Microsoft.Agents.AI.Foundry.Hosting, which partitions
    per user; the separate type of the same name in the framework's own hosting package is
    untouched. GetSessionAsync changes its return type and its meaning. The type is public, but
    the handler is its only caller and the package is still in preview, so both in-box stores are
    updated here and nothing else has to follow.

  • What do you want reviewers to focus on?

Whether refusing a session that carries a conversation id is the right call, or whether such a
container should instead be allowed to run with its service holding the conversation and the
handler standing down entirely.

Related Issue

N/A

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

The handler used to fetch the platform conversation history and prepend it to the
input of every turn. For a ChatClientAgent that runs in parallel with its own chat
history provider, so the conversation had two sources at once. It also had a hidden
cost: platform items carry no chat-history source marker, so the agent's provider
stored them again as if this turn had written them, leaving a second copy of the
conversation inside the persisted session that then diverges from the platform.

Make the chat history provider the single source for a ChatClientAgent:

- Add FoundryChatHistoryProvider, which reads the conversation through
  ResponseContext.GetHistoryAsync (it already resolves previous_response_id and the
  conversation the request belongs to) and stores nothing, because the platform
  persists the response items itself. An instance is created per request because it
  holds that request's context, and it is passed as a run-scoped override so the host
  does not have to mutate the agent.
- Register it only when the agent was created without a chat history provider. When
  one was supplied at construction, that provider owns the conversation and the
  platform history is not used at all.
- Stop adding the platform history to the input for a ChatClientAgent, since the
  provider now delivers it.

A workflow hosted as an agent is not a ChatClientAgent and has no provider pipeline,
so it keeps receiving the platform history from the handler exactly as before.
Cover the three symptoms the previous handler produced, each verified to fail when
the handler is reverted to fetching the platform history into the turn input:

- the conversation the service already keeps was copied into the persisted agent
  session by the default in-memory history provider;
- a custom history provider was asked to write that same conversation into its own
  database, because platform items carry no chat-history source marker and so look
  like content this turn produced;
- an agent with its own provider received both that provider's history and the
  platform's in a single request.

Also state precisely, in the provider's remarks, why nothing is written back: for a
stored request the response orchestrator hands the finished response to its responses
provider, which persists the input and output items that a later turn then reads back
through GetHistoryAsync; for a non-stored request nothing is persisted and nothing is
readable, so the request is self-contained either way.
A conversation can mix turns the service stores with turns it does not. History is
resolved from previous_response_id or the conversation regardless of the current
request's store flag, so an unstored turn still reads the stored ones back, but the
service records nothing for it and a later turn would never see it again.

Reading the platform history through FoundryChatHistoryProvider alone lost those
turns: from the second turn onwards the handler treats the session as a resume and
stops feeding history in, and the provider kept nothing of its own, so an unstored
turn simply vanished from the conversation. A regression test drives three turns of
one conversation, the first stored and the rest not, and without this change the
model receives only [second question, ok, third question]: the stored opening turn
is gone.

Give the provider both halves instead of choosing one:

- reading returns what the service serves, followed by the turns kept in the session,
  which are by definition later than anything the service recorded;
- writing keeps a turn only when the service was not asked to store it, so a stored
  turn is never duplicated and an unstored one is never lost.

The turns are held in the agent session under the provider's own state key, so they
travel with the session the host already persists.
A conversation can move between stored and unstored turns, and the unstored ones live
only in the agent session. Going back to a stored turn after that would have the
service record it on top of turns the service never saw, so anyone reading the
conversation back from the service would find an answer with no question. Refuse it
before the model is called instead of writing that gap.

Cover the whole shape with a walkthrough of nine turns over one conversation and three
provider instances, each with its own session:

- an instance that never took an unstored turn starts from the turn the service last
  saved, and does not see another instance's unstored turns;
- an instance that did keeps reading the saved turns and adds its own on top;
- asking such an instance for a stored turn is refused, twice, while unstored turns
  keep working;
- a turn stored from one instance does not appear for another, because it sits on a
  different branch of the conversation and so is not among the turns leading to what
  that other instance last saved.
The turns the service was not asked to store are written into the agent session's state
bag under this provider's own state key, and a new provider is built for every request,
so nothing is held on the provider object itself. The walkthrough named its three
threads after provider instances, which read as if the object carried the memory.

Name them after the sessions they are, and add a test that pins the behaviour down: a
turn kept through one provider object is read back by a different one given the same
session, and is absent for one given another session.
The session decides what is kept, but the provider still decides two things: which
service-side conversation is read, because it holds the request's response context, and
whether the turn is kept at all, because it holds the request's store flag.

Add two tests that separate those from the session:

- two providers reading one session, each built for a request of a different
  conversation, return the same kept turn behind different served turns;
- two providers writing to one session, one for a stored request and one for an
  unstored one, leave only the unstored turn behind.
The comment stated that a workflow hosted as an agent has no provider pipeline
without saying what that means. It derives from AIAgent directly, so it never calls
a ChatHistoryProvider and does not read the run options' additional properties: the
provider could not reach it even if it were registered.
The handler decided that a turn was resuming an existing conversation by looking for
state on the session. That reading broke once the handler itself started writing to the
session before the check: it records the caller's identity there, so a session created
moments earlier already carried state and the very first turn of a conversation looked
like a resume. Its history was then never fetched, and the agent answered knowing
nothing of a conversation the service was already holding. It only showed up when
hosted, because running locally there is no identity to record.

Let the store answer the question instead. GetSessionAsync now returns null when nothing
is stored rather than quietly handing back a new session, so a non-null result means a
prior turn established this session and nothing else has to be inferred. Callers that
just want a usable session can use the new GetOrCreateSessionAsync, which is written in
terms of GetSessionAsync so a store overriding one gets the other for free.

Both store implementations and their tests follow the plain-lookup contract: a miss
creates nothing, deserializes nothing, and touches no directory.
FoundryChatHistoryProvider is internal, so the attribute reached no caller: the marker
exists to warn people consuming the public surface. It also does not follow from the base
type, which does not carry one, and most internal types in this package have none either.
Removing it leaves two usings behind, so they go as well.
An agent refuses a second history manager once the model reports a conversation id of its
own, which happens as soon as the container lets the model keep the conversation. The
guard is meant for an application that configured a provider by hand and would otherwise
end up with two of them. Here the host is the one supplying the provider, deliberately and
for every turn, so the guard was rejecting the arrangement it is hosting: the first turn
failed while streaming, and every later one failed before reaching the model at all.

Turn the three conflict settings off on the agent the host is serving, and let the
provider decide what reaches the model. A test drives two turns of one conversation
against a model that reports a conversation id and asserts both complete.
A request asking the hosting service not to store the response was honoured there and nowhere else, so the service behind the agent's own chat client kept recording the conversation and reporting an id for it. A caller opting out of storage still ended up with a stored conversation, and the container went on continuing it.

Only that direction travels. Carrying store=true across would either force storage on a container whose author turned it off on purpose or change nothing, since storing is already the default.
The host no longer supplies a chat history provider of its own. It writes the turns the service holds into the provider the agent already created for itself, and only when that is the stock in-memory one, so an agent given a provider keeps sole control of its storage and the model receives the conversation once.

A conversation the caller stops asking the service to store moves into the session state and stays there. The session's conversation id no longer names anything the service records and cannot be cleared, so the session is cloned without it on that single turn. Asking for a stored turn afterwards is refused: the service would record a turn whose predecessors it does not hold.

An agent that does not read history through a provider, a hosted workflow for example, is still given its prior turns as input, now marked as chat history so no provider along the way stores them as new.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The store=false propagation via ChatOptions.RawRepresentationFactory can suppress agent/container RawRepresentationFactory behavior due to ChatClientAgent’s factory chaining semantics, risking incorrect request shaping.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

This PR adjusts the Foundry hosted-agent request pipeline to ensure each turn’s conversation history comes from exactly one place (service history, provider/session state, or the model’s own conversation id), preventing duplicate replay/storage and fixing incorrect “resume” detection.

Changes:

  • Update hosted session store semantics so GetSessionAsync is a pure lookup returning null when nothing is persisted, and add GetOrCreateSessionAsync for callers that want a usable session.
  • Rework AgentFrameworkResponseHandler history routing: preload service history into the default InMemoryChatHistoryProvider for ChatClientAgents, stamp replayed service history as ChatHistory when passed as input, and use store presence (not session state) as the resume signal.
  • Propagate store=false into chat-client options via CreateResponseOptions.StoredOutputEnabled = false.
File summaries
File Description
dotnet/tests/Microsoft.Agents.AI.Foundry.Hosting.UnitTests/FileSystemAgentSessionStoreTests.cs Updates tests to reflect GetSessionAsync now returning null on cache miss and adds coverage for GetOrCreateSessionAsync.
dotnet/tests/Microsoft.Agents.AI.Foundry.Hosting.UnitTests/AgentFrameworkResponseHandlerTests.cs Adds regression coverage for resume detection, single-source history routing, and stored/unstored conversation continuity.
dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/InputConverter.cs Attempts to forward store=false to the underlying chat client via RawRepresentationFactory.
dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/InMemoryAgentSessionStore.cs Makes GetSessionAsync a pure lookup returning null when not found.
dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/HostedSessionJsonUtilities.cs Adds a serialization shape to clone a ChatClientAgentSession without a conversationId.
dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/FileSystemAgentSessionStore.cs Makes GetSessionAsync return null on missing/empty session file.
dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/AgentSessionStore.cs Updates GetSessionAsync contract to nullable and introduces GetOrCreateSessionAsync.
dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/AgentFrameworkResponseHandler.cs Centralizes conversation ownership per turn, fixes resume detection, and prevents history duplication/storage.
Review details
  • Files reviewed: 8/8 changed files
  • Comments generated: 1
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Comment thread dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/InputConverter.cs Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 2 | Confidence: 57% | Result: All clear

Reviewed: Correctness, Test Coverage


Automated review by rogerbarreto's agents

ChatClientAgent chains a request's raw representation factory with the agent's by taking the agent's only when the request's returns null. The factory added for an unstored turn always answers, so anything the container configured on the agent's ChatOptions was silently dropped for that turn.

The agent's factory is now invoked first and its result is what carries the setting. A result that is not a CreateResponseOptions belongs to some other chat client, which has no notion of storing a response, so it is handed back untouched.
Comment thread dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/InputConverter.cs Outdated
Comment thread dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/AgentFrameworkResponseHandler.cs Outdated
The AgentServer SDK records a hosted turn through its own storage provider, around the handler, and serves the conversation back through ResponseContext.GetHistoryAsync. Anything the container stores of its own is a second conversation that storage provider never sees and no one reconciles.

The handler now takes that history as the single source and hands it to the agent as input alongside this turn's messages. The agent's own provider is replaced for the run by one holding its messages in a field, so a run that calls tools still has what its earlier calls produced while nothing survives the request. The service behind the agent's chat client is asked not to store on every turn, whatever the caller asked of the hosting service.

A session that still carries a conversation id means that service is recording a second conversation regardless, so the turn is refused with a 400 rather than run against something nobody can reconcile.
Withholding the conversation from every agent that is not a ChatClientAgent assumed they all carry it in their own session. A hand-written one that keeps nothing would answer with no history from its second turn on, so the check is now on the session type a workflow runs with, which is what actually accumulates the turns.

The conversation and previous response id tests went with it: the session key falls back to the partition of a freshly minted response id, which never has a session saved for it, so a loaded session already implies one of the two was sent.

Also asks a Chat Completions client not to store, since the setting carries the same name on both OpenAI request shapes.
The AgentServer SDK's storage provider records every hosted turn around the handler, and
that record is the conversation the caller reads. The agent's own run inside the container
talks to its own service, and when that service is asked to keep the turn it writes a
second copy of the same exchange, on a trail of its own that nobody reads and nobody
reconciles. The caller's conversation looks clean, so the second copy goes unnoticed.

The new downstream-store scenario runs an ordinary Foundry ChatClientAgent, like the first
hosted agent sample, wrapped so that after the run it appends DOWNSTREAM_ID=<id> to the
reply, carrying whatever its own run left behind. The tests then go looking for that id on
the service: finding it means a second copy exists.

Verified live against a Foundry project. On main both tests fail, reporting a readable id
such as resp_0940e276..., and here the container reports DOWNSTREAM_ID=none and both pass.
The run options were setting the conversation on every call, which the session already does.
The single turn test now binds the session to the conversation up front, and the multi turn
test starts from the agent's own default session and reads back what the hosted agent kept
for the caller off ChatClientAgentSession once the first turn returns.

Re-verified live: still fails on main, reporting a readable id such as resp_0c07a5e4..., and
still passes here.
Comment thread dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/InputConverter.cs
@rogerbarreto

Copy link
Copy Markdown
Member Author

Thanks for the review, I will be creating a follow up PR with the suggestions, merging this as a good progress, to unblock subsequent other works. Thanks!

@rogerbarreto
Roger Barreto (rogerbarreto) added this pull request to the merge queue Aug 6, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks Aug 6, 2026
@rogerbarreto
Roger Barreto (rogerbarreto) added this pull request to the merge queue Aug 7, 2026
Merged via the queue into microsoft:main with commit 18ceb18 Aug 7, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs .NET Usage: [Issues, PRs], Target: .Net

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants